A Simple Model for Neural Computation with Firing Rates and Firing Correlations

A simple extension of standard neural network models is introduced which provides a model for neural computations that involve both firing rates and firing correlations. Such an extension appears to be useful since it has been shown that firing correlations play a significant computational role in many biological neural systems. Standard neural network models are only suitable for describing neural computations in terms of firing rates. The resulting extended neural network models are still relatively simple, so that their computational power can be analysed theoretically. We prove rigorous separation results, which show that the use of firing correlations in addition to firing rates can drastically increase the computational power of a neural network. Furthermore, one of our separation results also throws new light on a question that involves just standard neural network models: we prove that the gap between the computational power of high-order and first-order neural nets is substantially larger than shown previously.